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Introduction 

This  essay  provides  a  sense  of  the  literature  written  specifically  about  intelligence  analysis, 
including  general  definitions,  themes,  divergent  views,  and  observations  on  gaps  in  literature 
coverage.  A  comprehensive  review  of  the  literature  indicates  that  while  much  has  been  written, 
largely  there  has  not  been  a  progression  of  thinking  relative  to  the  core  aspect  and  complexities 
of  doing  intelligence  analysis;  hence,  the  title,  “Once  Again,”  with  all  its  connotations.  The 
essay  focuses  on  the  core  aspect  of  analysis,  the  determination  of  meaning.  After  a  brief  review 
of  Sherman  Kent’s  comprehensive  work,  the  role  of  mental  models  in  determining  meaning  is 
discussed,  followed  by  recurring  alternative  views  of  analysis  processes  and  their  chronologies. 
Key  elements  of  the  analysis  processes  are  then  discussed,  including  hypotheses,  infonnation 
research,  and  the  marshalling  of  evidence,  including  how  they  influence  the  determination  of 
meaning.  The  literature  on  rigor  in  intelligence  analysis  is  reviewed,  including  the  role  of 
structured  analytic  techniques  and  automated  tools.  The  essay  concludes  with  observations  on 
the  state  of  the  literature. 


1 


Hundreds  of  works  have  been  written  on  intelligence.  Many  of  these  works  at  least 
touch  on  the  subject  of  intelligence  analysis.  However,  while  still  a  large  body  of  work,  it  is  a 
considerably  smaller  set  that  specifically  addresses  the  core  aspect  of  intelligence  analysis.  By 
necessity,  the  literature  represented  in  this  essay  is  a  much  smaller  set  than  that  reviewed  in  its 
preparation.  Consequently,  the  same  can  be  said  of  this  essay  as  Platt  (1957,  p.  xvii)  indicated 
about  his  classic  book  on  strategic  intelligence,  quoting  Vannevar  Bush,  “Everything  I  say  would 
be  contested  in  some  quarters.” 

Background 

What  is  intelligence  analysis?  The  lack  of  an  agreed  upon  definition  is  hardly  surprising  given 
the  lack  of  agreement  on  the  term  intelligence.  The  recent  efforts  of  Warner  (2002)  and  Johnston 
(2005,  p.  4)  to  establish  a  definition  of  intelligence  are  part  of  a  never  ending  effort.  Many  of  the 
definitions  of  intelligence  encompass  specific  aspects  of  intelligence  analysis.  Intelligence 
analysis  has  been  described  in  terms  of  the  sources  and  classifications  of  infonnation,  processes, 
purposes,  individual  and  organizational  efforts,  and  consumers.  Common  themes  include  the  use 
of  single  or  all  sources  of  infonnation,  use  of  denied  or  secret  infonnation  (although  most  find 
this  too  limiting),  discerning  pertinent  information,  making  inferences  and  judgments, 
developing  knowledge  individually  and  collectively  of  cunent  and  potential  threats  to  national 
security,  and  providing  useable  and  actionable  information  for  the  military  and  government  and 
non-government  policymakers.  Most  descriptions  of  the  functions  and  purpose  of  intelligence 
analysis  also  contain  elements  that  are  consistent  with  the  standard  definitions  of  both  analysis 
and  synthesis  (e.g.,  Webster’s  New  World  Dictionary,  Third  College  Edition): 
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“Analysis:  a  separating  or  breaking  up  of  any  whole  into  its  parts,  esp.  with  an 
examination  of  these  parts  to  find  out  their  nature,  proportion,  function,  interrelationship, 
etc.” 

“Synthesis:  the  putting  together  of  parts  or  elements  so  as  to  form  a  whole” 

With  these  definitions  neither  the  source  of  the  information  nor  the  purpose  of  the 
analysis  or  synthesis  defines  or  limits  the  scope  of  intelligence  analysis.  The  simplistic 
definition  then  is  that  intelligence  analysis  is  analysis  done  by  intelligence  organizations. 

Understanding,  knowledge,  and  the  need  to  determine  meaning  from  information  are 
inherent  in  the  preceding  definitions.  Indeed  the  essence  of  intelligence  analysis  is  detennining 
the  meaning  of  information  to  develop  knowledge  and  understanding.  Many  authors  have  rightly 
pointed  out  that  this  aspect  of  analysis  takes  place  in  broader  organizational  and  social  contexts. 
The  importance  of  viewing  analysis  holistically  in  these  contexts  as  work  done  by  individuals 
and  groups  with  all  of  their  respective  attributes  cannot  be  understated.  Other  Compendium 
essays  cover  many  of  these  aspects. 

The  meaning  derived  from  the  analysis  is  used  to  address  many  different  types  of 
questions,  which  are  categorized  in  variety  of  ways.  A  general  classification  of  the  questions, 
sometimes  described  as  types  of  intelligence  or  analysis,  includes:  strategic;  tactical  or  combat; 
current;  and  indications  and  warning.  These  classifications,  with  various  excursions  in 
tenninology,  have  been  relatively  stable  over  time.  Analysis  may  also  be  based  on  a  single 
source,  a  combination  of  several  sources,  or  consideration  of  all  sources  of  information.  One 
might  expect  the  literature  would  discuss  the  processes  and  attributes  associated  with  the  various 
types  of  analysis.  Generally,  this  is  not  the  case.  The  intelligence  literature,  with  few 
exceptions,  suggests  that  processes,  procedures,  and  cognitive  attributes  of  analysis  are  the  same 
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for  all  types  of  analysis.  Certainly,  some  commonality  exists  among  the  different  types  of 
analysis,  but  the  major  difference  is  the  complexity  of  the  analysis. 

In  the  late  1940s,  the  term  strategic  intelligence  entered  the  active  discourse.  Kent  (1949, 
p.  xi)  acknowledges  Pettee’s  (1946)  book  as  a  trail  breaker  in  the  literature  of  strategic 
intelligence.  Both  Kent  (1949)  and  Platt  (1957)  highlight  the  challenging  character  of  strategic 
intelligence:  it  is  vital  for  national  survival;  draws  important  conclusions  from  inadequate 
premises;  concerns  the  future  and  its  possibilities  and  probabilities;  addresses  the  problems  of 
estimating  what  other  human  beings  can  and  will  do  across  a  broad  band  of  the  future; 
encompasses  a  large  and  diverse  substantive  content,  thus  requiring  group  effort;  necessitates  the 
use  of  specialized  techniques,  considerably  extending  the  expertise  needed;  and  is  done  on  a 
comprehensive  scale,  “the  world  is  literally  its  province.”  (Platt  1957,  p.  16)  All  of  these 
characteristics  increase  the  complexity  of  the  work.  While  other  types  of  analysis  share  some  of 
these  characteristics,  strategic  intelligence  analysis  encompasses  all  the  cognitive  and 
infonnation  complexities  associated  with  analysis.  Thus,  to  represent  the  full  complexities  of 
analysis,  this  representation  of  the  literature  concentrates  on  analysis  to  fulfill  the  need  for 
strategic  intelligence. 

In  the  Beginning . Sherman  Kent 

Sherman  Kent’s  1949  book,  Strategic  Intelligence  for  American  World  Policy,  is  the  seminal 
publication  for  describing  and  explaining  the  processes  and  attributes  of  strategic  intelligence. 
Many  consider  it  the  foundation  for  all  subsequent  research  and  publications  on  intelligence 
analysis.  This  is  best  illustrated  by  Professor  Ernest  May,  a  leading  authority  on  the  history  of 
intelligence,  in  his  1989  remarks.  When  asked  why  he  continued  to  quote  from  Kent’s  book 
written  40  years  earlier,  despite  the  numerous  books  published  since,  Professor  May  replied, 
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“Yes,  but  with  concept  after  concept,  in  40  years  nobody  has  ever  stated  things  as  smartly  as 
Kent.”  (Davis  2002,  p.  7) 

Kent  deals  with  nearly  all  aspects  of  intelligence.  He  begins  “Intelligence  is 
Knowledge.”  (Kent  1949,  p.  1)  To  set  the  stage  for  his  discussion  of  the  analysis  process,  he 
indicates: 

“. .  .intelligence  activity  consists  basically  of  two  sorts  of  operation. .  .the  surveillance 
operation ,  by  which  I  mean  the  many  ways  by  which  the  contemporary  world  is  put 
under  close  and  systematic  observation,  and  the  research  operation.  By  the  latter  I  mean 
the  attempts  to  establish  meaningful  patterns  out  of  what  was  observed  in  past  and 
attempts  to  get  meaning  out  of  what  appears  to  be  going  on  now.  The  two  operations  are 
virtually  inseparable...”  (Kent  1949,  p.  4)  (emphasis  added) 

“. .  .research  is  a  systematic  endeavor  to  get  firm  meaning  out  of  impressions. .  .Research 
is  the  only  process  which  we  of  the  liberal  tradition  are  willing  to  admit  is  capable  of 
giving  us  the  truth,  or  a  closer  approximation  to  truth. .  .truth  is  to  be  approached,  if  not 
attained,  through  research  guided  by  systematic  method.  In  the  social  sciences  which 
very  largely  constitute  the  subject  matter  of  strategic  intelligence,  there  is  such  a  method. 
It  is  much  like  the  method  of  physical  sciences.  It  is  not  the  same  method,  but  it  is  a 
method  none  the  less.”  (Kent  1949,  pp.  155-156)  (emphasis  added) 

Kent  (1949)  identified  three  classes  of  substantive  content:  descriptive,  current 
reportorial,  and  speculative-evaluative.  He  describes  the  descriptive  as  “. .  .basic  to  the  other 
two. .  .the  groundwork  which  gives  meaning  to  day-to-day  change  and. .  .without  which 
speculation  into  the  future  is  likely  to  be  meaningless.”  (Kent  1949,  p.  1 1)  Current  reportorial  is 
described  as  “Keeping  track  of  the  modalities  of  change...”  (Kent  1949,  p.  30)  Knowledge 
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which  is  .  .far  more  speculative  than. .  .the  basic  descriptive  and  current  reportorial. . .”  and 
“. .  .puts  a  very  high  premium  on  the  seeker’s  power  of  evaluation  and  reasoned  extrapolation. . .” 
is  called  speculative-evaluative.  (Kent  1949,  pp.  39-40) 

Like  few  others,  Kent  ( 1 949)  covers  the  spectrum  of  issues  associated  with  analysis  in  a 
comprehensive,  holistic  way.  He  addresses  intelligence  as  both  organization  and  activity.  While 
beyond  the  scope  of  this  essay,  his  concerns  about  intelligence  producer  and  consumer 
relationships  and  issues  of  integrity  and  objectivity  have  been  discussed  extensively  in  the 
literature.  Kent’s  thoughts  about  analysts’  views  impacting  the  analysis,  analytic  processes,  and 
the  effects  of  short-tenn  consumer  questions  on  long-term  research  are  especially  relevant  to  this 
essay.  Later  sections  discuss  these  topics  and  examine  them  along  with  the  subsequent  literature 
on  intelligence  analysis,  beginning  with  the  role  of  what  Kent  (1949,  p.  199)  referred  to  as 
“patterns  of  thought.” 

Mental  Models  and  Determining  Meaning 

The  intelligence  literature  has  long  reflected  an  appreciation  that  determining  meaning  is 
influenced  by  the  analyst’s  mindset,  mental  model,  or  frame  of  mind.  Kent  (1949,  p.  199) 
indicated  “. .  .an  intelligence  staff  which  must  strive  for  reasoned  and  impartial  analysis. .  .has  its 
own  difficulties  with  view,  position,  slant,  and  line. .  .it  is  made  up  of  men  whose  patterns  of 
thought  are  likely  to  color  their  hypotheses  and  whose  colored  hypotheses  are  likely  to  make  one 
conclusion  more  attractive  than  the  evidence  warrants.” 

Heuer  (1999,  p.  10),  among  others,  has  also  written  extensively  about  the  relationship 
between  an  analyst’s  mindset  and  his  or  her  interpretation  of  infonnation,  indicating  that  a 
mindset  is  a  screen  or  lens  through  which  one  perceives  the  world.  Davis’  (1992,  p.  13) 
characterization  of  mindset  as  the  “...distillation  of  the  intelligence  analyst’s  cumulative  factual 
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and  conceptual  knowledge  into  a  framework  for  making  estimative  judgments  on  a  complex 
subject”  is  the  tenor  of  the  literature.  However,  while  there  is  a  general  appreciation  of  the  role 
of  mindset,  all  the  various  factors  that  fonn  and  change  one’s  mindset  (mental  model)  are  less 
well  represented  in  the  intelligence  literature. 

Context  is  one  of  the  factors  that  forms  and  changes  mental  models.  “It  is  the  context  of 
the  situation  alone  which  gives  point  and  meaning  to  the  subsequent  elements  of  the 
speculation,”  writes  Kent  (1949,  p.  45).  Similarly  Heuer  (1999,  p.  41)  writes,  “The  significance 
of  infonnation  is  always  a  joint  function  of  the  nature  of  the  information  and  the  context  in 
which  it  is  interpreted.  The  context  is  provided  by  the  analyst  in  the  form  of  a  set  of  assumptions 
and  expectations  concerning  human  and  organizational  behavior.” 

More  recently,  Patterson,  Roth,  and  Woods  (Patterson  et  al.  1999  &  2001)  have  written 
extensively  on  the  importance  of  context  relative  to  the  problem  being  solved,  specifically  as  it 
relates  to  intelligence  analysts  determining  meaning.  In  a  more  recent  work  Woods  et  al.  (2002, 
p.  27),  state,  “. .  .the  significance  of  a  piece  of  data  depends  on:  other  related  data;  how  the  set  of 
related  data  can  vary  with  larger  context;  the  goals  and  expectations  of  the  observer;  the  state  of 
problem-solving  process  and  stance  of  others.”  Importantly,  this  statement  highlights  the  role  of 
purpose  (the  problem  to  be  solved)  in  the  detennination  of  meaning. 

In  addition  to  the  role  of  context  and  purpose  in  shaping  the  mental  model,  other  factors 
are  recognized  such  as  past  experience,  education,  cultural  values,  role  requirements, 
organizational  norms  and  the  specifics  of  the  information  received  (Heuer  1999,  p.  xxi).  Given 
the  importance  of  the  mental  model  in  influencing  and  shaping  the  analysis  (i.e.,  from  problem 
exploration  and  formulation,  to  purpose  refinement,  through  data  acquisition  and  evaluation,  and 
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ultimately  determining  meaning  and  making  judgments),  it  is  not  surprising  how  it  influences  the 
discussion  of  intelligence  analysis. 

Many  criticisms  of  intelligence  analysis,  including  those  in  retrospective  studies  of 
intelligence  failures,  center  on  the  negative  aspects  of  mindset  or  analysts’  use  of  mental  models. 
These  criticisms  include  the  lack  of  “objectivity”  in  interpretation,  the  prevalence  of  many  forms 
of  bias  as  explicated  in  the  scientific  literature  and  in  Heuer’s  work  (1999,  pp.  1 1 1-171),  and  the 
lack  of  rigor.  The  presence  of  inappropriate  collective  mental  models  is  also  central  to  the 
concerns  expressed  about  groupthink. 

Based  on  these  concerns  and  criticisms,  many  techniques  for  various  stages  of  the 
intelligence  process  have  been  recommended  to  improve  intelligence  analysis;  however, 
evaluation  is  needed.  Such  techniques  include  devil’s  advocacy,  red  teaming,  linchpin  analysis, 
various  modeling  techniques,  and  collaborative  approaches  such  as  wikis  and  blogs,  etc. 
(‘Improving  Intelligence  Analysis’  2007,  p.  31).  Yet,  the  scientific  literature  suggests  much 
more  complexity  is  involved  in  ameliorating  potentially  negative  effects  of  mental  models  than 
merely  application  of  a  specific  technique,  which  may  have  limited  efficacy  outside  the 
boundaries  and  conditions  under  which  empirical  studies  were  done. 

Mental  models  also  have  positive  effects  since  they  are  based  on  past  experience, 
education,  and  expertise,  resulting  in  a  larger  repertoire  of  reasoning  strategies  (‘Improving 
Intelligence  Analysis’  2007,  p.  46).  The  advantages  of  having  well-formed,  expansive  mental 
models,  however,  are  rarely  mentioned  in  the  intelligence  literature. 

Although  recommended  by  Heuer  (1999,  p.  178),  there  has  been  little  systematic  research 
on  the  application  of  these  and  other  techniques  to  intelligence  analysis  based  on  the  extensive 
scientific  findings,  and  few  scientific  studies  have  been  focused  specifically  on  intelligence 
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analysis.  Given  the  role  of  mental  models  in  determining  meaning,  the  lack  of  treatment  in  the 
intelligence  literature  is  a  significant  gap.  Research  is  needed  to  apply  the  diverse  and  systematic 
research  of  cognitive  psychology  and  other  fields  on  how  mental  models  are  fonned  and 
changed;  and  to  understand  the  components  of  mental  models,  how  to  leverage  the  positive 
effects  of  mental  models,  and  how  to  avoid  pitfalls  and  ameliorate  the  negative  effects  on 
intelligence  analysis. 

Data-Driven  Analysis,  Conceptually-Driven  Analysis,  and  the  Mosaic  Theory  of  Analysis 
The  nature  and  role  of  the  mental  model  is  central  to  Heuer’s  (1999,  p.  59-62) 
descriptions  of  “Data-Driven  Analysis,”  “Conceptually  (sic)  Driven  Analysis,”  and  the  mosaic  or 
jigsaw  puzzle  analogy,  which  Heuer  calls  the  “Mosaic  Theory  of  Analysis.”  These  tenns,  first 
published  in  1979,  are  frequently  mentioned  in  the  literature. 

Data-Driven  and  Conceptually-Driven  Analysis 

Heuer  refers  to  these  as  “types  of  analysis.”  Other  authors  have  not  elaborated  on 
descriptions  of  Data-Driven  and  Conceptually-Driven  Analysis.  However,  at  times  both  are 
referred  to  as  descriptions  of  the  analysis  process,  with  subsequent  authors  equating  Data-Driven 
to  bottom-up  or  evidence-based  processes  and  Conceptually-Driven  to  top-down  or  hypotheses- 
driven  processes,  which  are  discussed  later  in  this  essay.  However,  Heuer ’s  main  points  are  not 
in  terms  of  the  analysis  process,  but  focus  on  the  nature  and  complexity  of  the  problem 
(purpose),  the  development  of  the  mental  model,  and  the  role  that  mental  model  plays. 

In  Heuer’s  (1999,  p.  59)  description  of  Data-Driven  Analysis,  it  is  used  for  analyzing 
subjects  where  rules  and  procedures  are  relatively  well  established,  most  elements  of  the  mental 
model  can  be  made  explicit,  there  is  agreement  on  the  appropriate  model,  and  there  are  relatively 
objective  standards  forjudging  the  quality  of  analysis.  “In  this  type  of  analysis,  accuracy 
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depends  primarily  upon  the  accuracy  and  completeness  of  the  available  data.”  In  other  words,  its 
use  is  appropriate  for  subjects  that  are  relatively  well  understood.  The  description  does  not 
indicate  the  sequence  or  process  of  analysis. 

In  contrast,  “. .  .at  the  opposite  end  of  the  spectrum  from  data-driven  analysis. . is 
Conceptually-Driven  Analysis  (Heuer  1999,  pp.  59-60).  It  is  used  for  questions  that  do  not  have 
neat  boundaries,  when  there  are  many  unknowns,  many  potentially  relevant  variables, 
imperfectly  understood  relationships  with  enormous  complexity  and  uncertainty,  little  tested 
theory,  no  agreed  upon  analytical  schema,  and  largely  implicit  mental  models.  “Other  analysts 
examining  the  same  data  may  well  reach  different  conclusions,  or  reach  the  same  conclusions  but 
for  different  reasons.  This  analysis  is  conceptually  driven,  because  the  outcome  depends  at  least 
as  much  upon  the  conceptual  framework  employed  to  analyze  the  data  as  it  does  upon  the  data 
itself.” 

Mosaic  or  Jigsaw  Puzzle  Analogy 

The  nature  and  role  of  the  mental  model  are  also  dominant  in  the  discussions  about  the 
utility  of  a  mosaic  or  jigsaw  puzzle  analogy.  It  is  a  frequently  used  and  debated  analogy  of  the 
analysis  process.  The  assumption  about  whether  or  not  the  analyst’s  mental  model  plays  a  role  is 
the  focus  of  the  debate. 

The  use  of  the  tenns,  jigsaw  puzzle  and  mosaic,  are  credited  as  originating  with  early 
Directors  of  the  Central  Intelligence  Agency  (CIA)  (Hilsman  1952,  p.  3).  Rear  Admiral  Roscoe 
H.  Hillenkoetter  (1948,  p.  5),  who  assumed  his  duties  as  CIA  Director  in  1947,  described  the 
intelligence  product  as  a  “. .  .giant  jig-saw  puzzle,  into  which  we  are  continually  trying  to  fit  the 
pieces.”  While  he  did  not  mention  the  tenn  “mosaic,”  Lt.  Gen.  Hoyt  S.  Vandenberg  (1947,  p. 
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498),  in  his  testimony  to  the  Senate  as  CIA  Director,  described  intelligence  in  terms  of 
completing  a  “picture.” 

Hilsman  (1952,  p.  4)  implies  criticism  of  Vandenburg,  stating  that  “Apparently  he 
believes  that  what  information  is  relevant  and  necessary  is  determined  only  by  the  facts 
themselves.”  Hilsman  indicates  the  implication  of  the  picture-puzzle  analogy  is  that  the  process 
of  analysis  is  “. .  .a  question  merely  of  assembling  facts  in  much  the  same  common-sense  way 
that  one  puts  together  a  broken  vase:  by  looking  at  the  pieces  and  seeing  how  they  fit.”  He 
(Hilsman  1952,  p.  15)  goes  on  to  say  that  “Facts  cannot  in  themselves  contain  self-evident  or 
obvious  answers,”  rejecting  the  analogy  and  implying  that  Hilsman  does  not  consider  that  the 
puzzle  is  viewed  with  the  benefit  of  a  mental  model. 

Like  Hilsman  (1952),  Heuer  also  rejects  the  jigsaw  puzzle  analogy,  stating  it  has  distorted 
understanding  of  the  analytic  process  (Heuer  1999,  p.  61).  Heuer’s  rejection  is  most  frequently 
cited  by  others  who  oppose  this  analogy.  He  describes  the  mosaic  theory  as  “. .  .small  pieces  of 
information  are  collected  that,  when  put  together  like  a  mosaic  or  jigsaw  puzzle,  eventually 
enable  analysts  to  perceive  a  clear  picture  of  reality.  The  analogy  suggests  that  accurate 
estimates  depend  primarily  upon  having  all  the  pieces,  that  is,  upon  accurate  and  relatively 
complete  information.”  Heuer  (1999,  p.  62)  also  indicates  that  “Accurate  estimates  depend  at 
least  as  much  upon  the  mental  model  used  in  forming  the  picture  as  upon  the  number  of  pieces  of 
the  puzzle  that  have  been  collected.”  Heuer’s  distinction  between  mosaic  analysis  and  both  data- 
driven  and  conceptually-driven  analysis  underscores  his  view  that  mental  models  are  not  used  in 
the  mosaic  approach. 

Despite  these  criticisms,  others  find  it  a  useful  analogy,  as  a  puzzle  with  missing  pieces 
and  other  variations.  For  example,  another  director  of  CIA,  William  E.  Colby  (Hirschfeld  1987, 
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pp.  37-38),  said  “The  intelligence  analysis  process  is  like  a  jigsaw  puzzle;  one  puts  all  the  little 
pieces  together. .  .you  see  the  picture  and  overall  design  long  before  you  have  fdled  in  the  last 
puzzle  piece. .  .The  excellent  work  of  our  analysts. .  .consists  of  putting  these  pieces  together. 

They  come  up  with  very  good  overall  designs  of  the  scene  without  necessarily  every  last  item.” 
Colby’s  view  contrasts  with  Heuer’s  that  all  the  pieces  are  needed. 

Recently  Boslough  (2005,  p.  1),  discussed  the  puzzle  analogy  as  useful  but  neglecting 
“. .  .that  the  problem  is  both  distributed  and  dynamic.”  Most  recently,  Johnson  (2007,  p.  2) 
asserted  it  is  “A  useful  metaphor  for  thinking  about  strategic  intelligence...” 

Despite  the  prolonged  use  of  the  jigsaw  puzzle  analogy,  there  is  only  one  description  in 
the  intelligence  literature  of  the  cognitive  processes  associated  with  it  (Thompson  et  al.  1984,  p. 
4-2).  It  emphasizes  the  roles  of  context  and  mental  models: 

“If  the  picture  of  the  completed  puzzle  is  provided  then  a  schema  already  exists  to  direct 
the  search  for  particular  pieces. .  .successful  completion. .  .merely  requires  the  matching  of 
individual  pieces  with  parts  of  the  given  picture.  If  no  picture  is  provided,  the  puzzle  is 
more  difficult. .  .One  must  examine  a  piece  or  group  of  pieces  for  context  clues  so  that  an 
initial  conceptual  model  can  be  developed.  This  model  then  directs  the  search  for 
particular  pieces,  some  of  which  might  lead  to  the  rejection  of  the  initial  model  and  the 
development  of  another  one,  in  a  feedback  loop.” 

Notably  this  description  is  suggestive  of  much  later  descriptions  about  the  analysis 
process  as  “sensemaking”  discussed  later  in  this  essay.  These  emphasize  the  mutual,  iterative 
relationship  between  the  frame  (the  puzzle  picture  if  provided  or  the  concept  of  the  picture  that 
must  be  developed)  and  the  data  (the  pieces  of  the  puzzle),  as  well  as  how  the  model  (frame) 
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“directs  the  search  for  particular  pieces”  (the  data).  Boslough’s  (2005,  p.  1)  description  also 
captures  the  dynamic,  evolving  nature  of  analysis  of  the  sensemaking  descriptions. 

Even  though  the  jigsaw  puzzle  analogy  has  been  rejected  by  many,  there  is  a  common 
belief  in  the  intelligence  literature  that  mental  models  are  used  and  that  the  facts  do  not  speak  for 
themselves.  There  is  also  a  common  belief  that  analysts  must  extract  meaningful  patterns  from 
the  data  available  (Knorr  1964,  p.  23;  Clarkson  1981,  p.  9).  Given  the  benefit  of  a  mental  model, 
it  is  not  evident  how  extracting  meaningful  patterns  to  fonn  a  picture  is  cognitively  or 
conceptually  different  than  forming  a  picture  from  puzzle  pieces. 

Analysis  Processes 
Analysis  and  Scientific  Methods 

Using  scientific  methods  in  intelligence  analysis  and  discussing  the  analytic  process  in  tenns  of 
scientific  methods  is  a  frequent,  recurring  theme  in  the  intelligence  literature.  The  belief  that 
using  “the  scientific  method  is  the  only  way  of  knowing  that  consistently  produces  reliable 
knowledge”  motivates  its  application  to  analysis  (Bruce  2005,  p.  4).  However,  there  is  no 
consistent  definition  of  the  scientific  method  in  the  literature.  For  some,  the  logical  reasoning 
underlying  the  scientific  method  is  based  on  the  principle  of  eliminating  hypotheses  (Heuer 
1981,  p.  76).  For  others  (Collier  2005,  p.  19),  the  steps  in  the  scientific  method  are: 

1 .  Develop  research  problem  or  question. 

2.  Review  existing  knowledge. 

3.  Develop  theoretical  framework  (model). 

4.  Develop  research  hypotheses. 

5.  Conduct  research  design. 

6.  Conduct  data  collection. 
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7.  Conduct  data  analysis. 

8.  Distribute  new  knowledge. 

Another  definition  of  the  scientific  method  is  “the  formulation  of  hypotheses,  collection 
of  relevant  data,  testing  and  evaluation  of  hypotheses,  and  the  logical  derivation  of  conclusions.” 
(Moore  2006,  p.  63) 

Analysis  Processes 

Frequently  Shennan  Kent  is  acknowledged  as  the  individual  who  first  proposed  modeling 
the  analysis  process  based  on  the  scientific  method,  especially  the  method  used  in  the  social 
sciences.  As  referenced  earlier,  Kent  instead  proposed  another  method  “specifically  designed” 
for  intelligence,  choosing  not  to  modify  the  methods  of  the  social  or  physical  sciences.  The 
essence  of  Kent’s  method  is  exploration  of  the  intelligence  problem,  data  collection  prior  to 
deliberate  hypotheses  generation,  data  evaluation,  more  data  collection,  followed  by  hypotheses 
generation.  Kent  describes  seven  steps  or  stages  (Kent  1949,  pp.  157-158): 

“1.  The  appearance  of  a  problem  requiring  the  attention  of  a  strategic  intelligence  staff. 

2.  Analysis  of  this  problem  to  discover  which  facets  of  it  are  of  actual  importance  to  the  U.S. 
and  which  of  several  lines  of  approach  are  most  likely  to  be  useful  to  its  governmental 
customers. 

3.  Collection  of  data  bearing  upon  the  problem  as  formulated  in  stage  2.  This  involves  a  survey 
of  data  already  at  hand  and  available  in  the  libraries  of  documentary  materials,  and  an  endeavor 
to  procure  new  data  to  fill  in  gaps. 

4.  Critical  evaluation  of  the  data  thus  assembled. 
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5.  Study  of  the  evaluated  data  with  the  intent  of  finding  some  inherent  meaning.  The  moment  of 
the  discovery  of  such  a  meaning  can  be  called  the  moment  of  hypotheses.  In  reality  there  is 
rarely  such  a  thing  as  one  moment  of  hypothesis  though  some  students  of  method,  largely  as  a 
convenience,  speak  as  if  there  were.  Nor  can  it  be  said  categorically  at  what  stage  in  the  process 
hypotheses  appear.  One  would  be  pleased  to  think  that  they  appeared  at  this,  the  respectable 
stage  5,  but  in  actual  practice  they  begin  appearing  when  the  first  datum  is  collected.  They  have 
been  known  to  appear  even  before  that,  and  they  may  continue  to  appear  until  the  project  is 
closed  out — or  even  after  that.  ( emphasis  added) 

6.  More  collecting  of  data  along  the  lines  indicated  by  the  more  promising  hypotheses,  to 
confirm  or  deny  them. 

7.  Establishment  of  one  or  more  hypotheses  as  truer  than  others  and  statement  of  these 
hypotheses  as  the  best  present  approximation  of  truth.”  This  stage  is  often  referred  to  as  the 
presentation  state  [sic]  N 

Kent’s  description  in  Stage  5  about  the  nature  of  hypotheses  provides  key  insights  on  his 
thinking  about  the  dynamic  nature  of  determining  meaning.  It  indicates  that  meaning  associated 
with  the  problem  at  hand  is  resident  at  the  start  of  the  project,  determined  during  problem 
exploration,  forms  and  changes  continuously  as  each  piece  of  data  is  reviewed,  evolves  during 
the  development  and  crystallization  of  various  hypotheses,  and  continues  to  be  adjusted  even 
after  the  project  is  done. 

In  his  description  of  the  analysis  process,  Platt  (1957,  p.  76)  has  two  criticisms  of  the 
many  renditions  of  the  scientific  method.  First,  they  do  not  start  with  a  general  survey  and  plan 
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of  the  whole  problem  and,  second,  they  lead  one  to  believe  that  it  is  .  .a  streamlined  succession 
of  one  logical  step  after  another. . rather  than  a  highly  iterative  process  with  many  forward  and 
feedback  loops.  Platt  (1957,  p.  82)  also  emphasizes  “. .  .that  each  step  when  taken  is  considered 
as  tentative .”  Because  new  knowledge  changes  one’s  interpretation  of  the  infonnation  and 
increases  understanding  of  the  issues,  what  is  done  at  each  step  is  changeable.  Platt’s  (1957,  pp. 
77-82)  seven  steps  are  much  like  Kent’s,  with  both  emphasizing  up  front  problem  exploration 
(enabling  purpose  refinement)  and  data  collection  prior  to  deliberate  hypotheses  generation,  as 
well  as  the  iterative  or  tentative  nature  as  one  goes  through  the  process  of  detennining  meaning. 

Numerous  other  brief  descriptions  of  the  analysis  process  also  appeared  between  1976 
and  1991  stating  data  collection  or  information  acquisition  occurs  before  hypotheses  generation 
(e.g.,  Clauser  &  Weir  1976,  p.  77;  Phelps  et  al.  1984,  p.  2;  Palmer  1991,  p.  35).  A  detailed 
description  of  the  analysis  process  equivalent  to  Kent’s  and  Platt’s  then  appeared  in  2005 
(RAND  2005,  p.  2). 

Subsequently  in  2006,  another  model  (Figure  1)  of  the  intelligence  analysis  process 
consistent  with  Kent’s  and  Platt’s  was  published,  especially  when  taking  into  consideration  the 
narratives  describing  their  seven  stages  or  steps.  This  model  was  derived  from  the  results  of  a 
cognitive  task  analysis  of  intelligence  analysts  (Pirolli  2006,  p.2;  Pirolli  &  Card  2006,  p.  3).  The 
overall  process  represented  in  the  model  is  described  as  “A  Notional  Model  of  Analyst 
Sensemaking,”  with  the  cognitive  task  analysis  indicating  that  the  bottom-up  and  top-down 
processes  shown  in  each  loop  are  “...invoked  in  an  opportunistic  mix.”  (Pirolli  &  Card  2006,  p. 
1-4). 
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3. Search  for  6. Search  for  9. Search  for  12. Search  15. Reevaluate 


Figure  1.  Notional  Model  of  Analyst  Sensemaking 

The  term  “sensemaking”  emerged  in  the  intelligence  literature  predominantly  in  the  last 
five  years  as  a  term  used  to  describe  the  analysis  process  (e.g.,  Fishbein  &  Treverton  2004; 
Cooper  2005).  Most  often  the  intelligence  literature  references  the  research  of  Gary  Klein  and 
Karl  E.  Weick.  In  a  recent  paper,  Klein  et  al.  (2007,  p.  1 14)  defines  sensemaking  .  .as  the 
deliberate  effort  to  understand  events,”  describing  the  elements  of  sensemaking  using  the  terms 
“data”  and  “frame.”  A  frame  is  “...an  explanatory  structure  that  defines  entities  by  describing 
their  relationship  to  other  entities.”  Klein  et  al.  (2007,  p.  118)  go  on  to  say  that  “The  data 
identify  the  relevant  frame,  and  the  frame  determines  which  data  are  noticed.  Neither  of  these 
comes  first.  The  data  elicit  and  help  to  construct  the  frame;  the  frame  defines,  connects,  and 
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filters  the  data.”  Hence,  it  invokes  the  processes  in  “an  opportunistic  mix”  as  identified  by 
Pirolli  and  Card  (2006)  and  has  striking  similarities  with  the  processes  described  by  Kent  (1949) 
and  Platt  (1957).  Sensemaking  provides  an  emergent  foundation  for  a  scientific  basis  for  what 
was  perhaps  intuitively  sensed  and  captured  by  Kent  and  Platt. 

In  contrast  with  the  early  models  of  Kent  and  Platt  that  emphasized  more  extensive 
problem  exploration  and  information  acquisition  or  research  prior  to  deliberate  hypotheses 
generation,  Heuer  elaborated  a  different  version  of  the  analysis  process  in  1980  (Heuer  1981,  pp. 
66-67).  He  proposed  an  eight  step  model  of  “an  ideal”  analytic  process,  emphasizing  early 
deliberate  generation  of  hypotheses  prior  to  information  acquisition: 

1 .  Definition  of  the  analytical  problem 

2.  Preliminary  hypotheses  generation 

3.  Selective  data  acquisition 

4.  Refinement  of  the  hypotheses  and  additional  data  collection 

5.  Data  intervention  and  evaluation 

6.  Hypotheses  selection 

7.  Continued  monitoring 

In  the  late  1980s,  the  process  described  by  Heuer  started  to  be  reported  by  others.  For 
example,  (1)  the  analysis  process  as  “...a  process  of  fonning  hypotheses,  testing  them  with  data, 
and  integrating  these  findings  into  explanations,  assessments,  or  predictions.”  (Berkowitz  & 
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Goodman  1989,  p.  85),  and  (2)  hypothesis  generation  as  the  first  step  of  the  analysis  process, 
followed  by  gathering  and  listing  evidence  and  other  steps  (United  States  1989,  pp.  106-116). 
Heuer’s  1980s  description  of  the  process  continues  to  be  a  view  of  the  analysis  process  as 
manifested  by  the  following  2002  Joint  Military  Intelligence  College  (JMIC)  chart  (Figure  2) 
(Waltz  2003,  p.  8). 


•Structure  assessment 
and  underscore  needs 
•Address  implications, 
dangers,  opportunities 
•Demonstrate  authority  to 
speak  on  issue 

•Research  methods 
•Firsthand  knowledge 
of  subject 
•All-source 
•Closeness  to 
collectors 

•Clear  articulation  of 
facts  &  assumptions 
•Ensure  the  right  question 
is  being  asked 
•Identify  trade-off  between 
quality  and  time 


•Identify  all  plausible 
hypotheses 
•Consult  internal  and 
external  expertise 
•In  brainstorming 
modes  suspend 
judgment 
•Reduce  list  to  to 
workable  number  of 
hypotheses  for  more 
detailed  analysis 
•Maintain  awareness  of 
possible  new 
hypotheses  emergent 
from  collection  and 
evidence  valuation 


•Make  use  of  classified 
information  to  provide 
insight 

•Depict  sources  of 
information  (e  g.  direct  and 
testimonial  evidence) 

•On  complex  matters, 
make  levels  of  confidence 
explicit 

•Contact  internal  and 
external  experts  for 
additional  information 
•Collect  information  on  all 
reasonable  hypotheses 
•Seek  evidence  to  refurte 
hypotheses 
•Identify  evidence  with 
greatest  diagnostic  value 
forjudging  relative 
likelihood  of  alternative 
hypotheses 


•Develop  arguments 
against  each  hypothesis 
•Use  matrix  to  keep  track 
of  evidence 
•Make  assumptions 
explicit 

•Analyze  sensitivity  of 
nconclusions 
•Guard  against  various 
cognitive  biases 


•Highlight  important 
findings  and  judgments 
•Make  actionable  via  policy 
implications 

•Emphasize  what  is  new, 
different  and  attention- 
worthy 

•Address  What  is  not 
known 

•Suggest  collection 

strategies  to  fill  gaps 

•Identify  trends,  patterns, 

precedents  of  dangers  and 

opportunities 

•Spell  out  rules  to  extend 

factual  base 

•Clarify  premises  and 

suppositions  that  underlie 

judgments 

•Identify  drivers  that  will 
determine  outcome 


Figure  2.  JMIC  Intelligence  Analysis  Process  Model 


Comparison  of  Analysis  Processes 

Based  on  the  preceding,  the  intelligence  literature  predominantly  contains  two  different 
interpretations  of  the  analysis  process,  which  exist  concurrently  in  the  literature.  Both  views 
agree  on  the  need  to  generate  multiple  hypotheses  that  must  be  subjected  to  verification  or 
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rejection,  but  they  differ  on  the  activities  that  should  ideally  precede  the  deliberate  generation  of 
hypotheses  and  the  information  used  to  generate  hypotheses,  thus  having  implications  for  the 
amount  and  scope  of  information  acquired  that  will  serve  as  potential  evidence.  These  two 
approaches  have  been  characterized  as  bottom-up  or  evidence-based  versus  top-down  or 
hypotheses-driven.  Differences  on  the  use  of  a  bottom-up  or  a  top-down  process  are  not  unique 
to  the  intelligence  literature.  In  the  scientific  literature,  arguments  date  back  to  Descartes  (1596- 
1650)  and  Newton  (1642-1727),  as  have  those  on  the  terms  “analysis”  and  “synthesis”  (Schum 
2006,  pp.  43-47).  Schum  (2006,  pp.  44)  points  out  “there  is  considerable  confusion  regarding 
the  use  of  the  terms  analysis  and  synthesis  in  connection  with  discovery-related  activities.” 

While  there  are  general  definitions  of  bottom-up  and  top-down  processes,  in  the 
intelligence  literature  the  two  processes  are  described  most  explicitly  by  Schum  (1987,  p.  52)  as 
inference  processes.  A  “bottom-up”  or  “data-driven”  inference  is  one  “. .  .in  which  reasoning 
proceeds  from  observable  evidence  to  possible  conclusions.”  A  “top-down”  or  “hypotheses- 
driven”  inference  process  is  one  “. .  .in  which  a  possible  conclusion  allows  us  to  generate  a  chain 
or  hierarchy  of  events  leading  to  one  or  more  potentially  observable  events  whose  occurrence  or 
nonoccurrence  would  be  evidence  bearing  upon  our  possible  conclusion.” 

Overall,  there  has  been  little  written  on  the  relative  merits  of  either  approach.  Cooper 
(Bodnar  2003,  p.  ix)  comments  favorably  that  Bodnar  recognizes  the  limitations  of  the  bottom-up 
or  evidence-based  approach:  “It  is  gratifying  that  he  also  appreciates  how  limited  the  evidence- 
driven  “connect  the  dots”  model  is  in  its  applicability  and  that  employing  other  models  that  rely 
more  on  hypothesis  generation  and  testing  are  essential.  Moreover,  although  Bodnar  lives  within 
an  ‘evidence-based  culture,’  he  stresses  the  need  to  provide  context  as  the  only  way  to  ensure 
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that  the  evidence  can  be  interpreted  correctly;  ‘the  facts’  don’t  speak  for  themselves.”  Cooper 
(2005,  p.  5)  later  reinforces  this  view  characterizing  an  evidence-based  approach  as  an 
. .  ‘Industrial  Age’  intelligence  production  model.” 

In  contrast,  Ryan  (2006,  p.  287)  is  quite  critical  of  the  top-down  or  hypotheses-driven 
approach  citing  several  examples  of  faulty  analysis  resulting  from  its  use,  including  the  2002 
National  Intelligence  Estimate  on  Iraqi  weapons  of  mass  destruction.  She  states,  “Rather  than 
constructing  a  ‘fact-based  hypothesis’,  the  Pentagon’s  Office  of  Special  Plans  (OSP),  followed  to 
a  significant  degree  by  the  CIA  particularly  in  the  pivotal  National  Intelligence  Estimate  (NIE)  of 
October  2002,  designed  an  (sic)  ‘hypothesis-based  analysis’,  which  started  with  a  preferred 
scenario  and  then  found  data  that  might  support  that.  In  effect,  it  was  a  worst-case  scenario 
presented  as  a  probability.”  Ryan  (2006,  p.  309)  also  states  that  “Such  serious  credibility  issues 
might  be  lessened  by  avoiding  hypothesis-based  analysis  which  lends  itself  well  to  extreme 
politicization  of  intelligence.” 

The  recent  discourse  about  sensemaking  explicitly  elaborates  the  two  predominant  views 
of  the  analysis  process,  bottom-up  and  top-down.  The  iterative  loops  in  the  sensemaking  model 
(Figure  1),  as  well  as  Klein’s  (2007)  data-frame  theory,  are  reminiscent  of  Kent  and  Platt.  As 
Fishbein  and  Treverton  (2004,  pp.  viii-ix)  describe  “Sense-making  is  a  continuous,  iterative, 
largely  intuitive  effort  to  paint  a  picture. . .”  to  cope  with  “. .  .high  levels  of  complexity  and 
uncertainty.”  Klein  et  al.  (2007,  p.  1 14)  point  out  that  “Practitioners  in  real-world  domains  must 
confront  and  understand  complex,  dynamic,  evolving  situations  that  are  rich  with  various 
meanings...  there  is  nothing  like  an  ‘endpoint’  since  the  understanding  of  dynamic  events 
requires  that  the  understanding  itself  be  dynamic,  not  a  stored,  frozen  ‘meaning’.”  Yet  from  a 
classical  scientific  viewpoint  there  is  discomfort  with  the  intuitive  aspects  of  sensemaking  as 
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Klein  (2004,  p.  hiv)  alludes  to  in  an  earlier  work.  Collier  (2005,  pp. 18-19)  describes  intuition 
with  its  “implicit  connection  of  facts”  as  a  “lower  order”  origin  of  knowledge  (in  his 
tenninology,  level  4  of  7),  with  science  being  the  highest  order. 

The  processes  most  frequently  used  by  analysts  are  unknown.  While  Pirolli  and  Card’s 
(2006)  preliminary  cognitive  task  analyses  indicated  both  are  used  opportunistically,  in  his 
classic  work  Heuer  (1999,  p.  95)  asserts  “The  way  most  analysts  go  about  their  business  is  to 
pick  out  what  they  suspect  intuitively  is  the  most  likely  answer,  then  look  for  the  available 
information  from  the  viewpoint  of  whether  or  not  it  supports  this  answer.”  Research  is  needed 
to  better  understand  the  processes  used  by  analysts  in  their  natural  work  environments  and  to 
understand  the  implications  of  various  processes  on  the  detennination  of  meaning. 

Hypotheses 

Definitions  of  Hypotheses 

As  described  previously,  two  processes  concurrently  exist  in  the  intelligence  literature  leading  to 
the  generation  of  hypotheses.  The  process  favored  detennines  both  the  timing  of  deliberate 
hypotheses  generation  and  the  character  of  the  hypotheses  generated.  That  is,  not  only  are  two 
paths  evident,  but  the  character  of  the  hypotheses  generated  is  different. 

Overall  there  is  agreement  on  the  definition  cited  by  McDowell  (1998,  p.94)  from  the 
Concise  Oxford  Dictionary:  “(1)  a  supposition  made  as  the  basis  for  reasoning,  without 
assumption  of  its  truth,  or  (2)  a  supposition  used  as  a  starting  point  for  further  investigation  from 
known  facts.” 

Specific  definitions  used  in  the  intelligence  literature  include  “...hypothesis  in  its 
broadest  sense  as  a  potential  explanation  or  conclusion  that  is  to  be  tested  by  collecting  and 
presenting  evidence.”  (Heuer  1999,  p.  32);  and  “...a  conclusion  or  assumption  you  may  reach 
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about  anything  at  all. .  .a  plausible  explanation.”  (McDowell  1998,  p.  93)  Kent’s  (1967,  p.  35) 
view  of  hypotheses  is  even  more  expansive,  “Every  time  the  mind  perceives  a  relationship 
between  two  things  it  creates  a  hypothesis.” 

In  the  intelligence  literature,  the  first  dictionary  definition  is  most  often  associated  with 
the  bottom-up  or  evidence-based  process  since  the  hypotheses  are  not  used  as  the  starting  point 
but  to  reason  about  the  acquired  evidence.  The  second  dictionary  definition  above  is  most  often 
associated  with  the  formation  of  hypotheses  that  represent  potential  answers  or  solutions  to  the 
intelligence  problem,  using  a  top-down,  hypotheses-based  approach.  As  stated  by  Heuer  (2005, 
p.  91)  “. .  .analysts  generally  start  by  identifying  what  appears  to  be  the  most  likely  hypothesis— 
that  is,  the  tentative  estimate,  explanation,  or  description  of  the  situation  that  appears  most 
likely.” 

Schum  (1987,  p.  29)  provides  the  most  detailed  overall  treatment  of  hypotheses  at  each 
step  in  the  analysis  process  in  his  discussion  of  evidence  and  inference.  His  principle  message  is 
that  marshalling  evidence  in  support  of  the  identified  hypotheses  results  in  a  hierarchical 
inference  network.  At  the  top  of  the  network  will  be  the  “major”  or  “ultimate”  hypotheses,  with 
various  levels  of  “indicators  or  intennediate  hypotheses”  identified  which  have  some  bearing  on 
the  major  hypotheses.  “By  such  a  process  you  are  essentially  determining  categories  of 
potentially-relevant  evidence.”  This  forms  the  hierarchical  structure  for  the  inference  problem, 
with  intennediate  hypotheses  at  each  level  supporting  the  major  hypotheses  at  the  top  of  the 
network. 

Hypotheses  Generation  and  Multiple  Hypotheses 

Generating  hypotheses  is  frequently  discussed  in  the  intelligence  literature.  Heuer  has  provided 
the  most  thorough  treatment  of  the  process.  Acknowledging  that  his  list  is  not  exhaustive,  Heuer 


23 


(1999,  p.  32)  provides  three  strategies  for  generating  hypotheses,  indicating  “No  one  strategy  is 
necessarily  better  than  the  others.”  His  strategies  are  “Situational  Logic,”  “Applying  Theory,” 
and  “Comparison  with  Historical  Situations.” 

Situational  Logic  is  described  as  “. .  .the  most  common  operating  mode. .  .Generation  and 
analysis  of  hypotheses  start  with...  concrete  elements  of  the  current  situation...  the  analyst  seeks 
to  identify  the  logical  antecedents  or  consequences  of  this  situation.  A  scenario  is  developed  that 
hangs  together  as  a  plausible  narrative.  The  analyst  may  work  backwards  to  explain  the  origins 
or  causes  of  the  current  situation  or  forward  to  estimate  the  future  outcome.”  (Heuer  1999,  p.  32- 
33)  In  Applying  Theory  a  “. . .  generalization  based  on  the  study  of  many  examples  of  some 
phenomenon. . .”  is  used,  and  “. .  .conclusions  are  judged  to  follow  from  a  set  of  conditions  and  a 
finding  that  these  conditions  apply  in  the  specific  case  being  analyzed.”  (Heuer  1999,  p.  34-35) 
When  using  Comparison  with  Historical  Situations  “An  analyst  seeks  understanding  of  current 
events  by  comparing  them  with  historical  precedents  in  the  same  country,  or  with  similar  events 
in  other  countries. .  .analysts  use  their  understanding  of  the  historical  precedent  to  fill  gaps  in 
their  understanding...”  (Heuer  1999,  p.  38) 

Heuer  (1999,  pp.  40-41)  discusses  a  fourth  strategy  called  “Data  Immersion,  ”  which  he 
rejects  for  the  same  reason  as  the  jigsaw  puzzle  analogy:  the  assumption  that  a  mental  model  is 
not  used  and  this  is  not  possible  for  analysts  to  do.  According  to  Heuer,  this  approach  is 
described  as  analysts  “. .  .immersing  themselves  in  the  data  without  fitting  the  data  into  any 
preconceived  pattern.”  Heuer’s  description  of  the  immersion  process  is  different  than  the 
description  used  in  the  qualitative  research  literature  (Borkan  1999).  While  both  descriptions 
include  an  intense  concentration  on  the  data,  the  qualitative  research  description  emphasizes  the 
role  of  the  person  engaged  in  the  process  and  that  success  is  dependent  on  the  full  involvement 
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of  the  individual.  As  described  in  the  qualitative  research  literature,  data  immersion  appears  to 
be  a  promising  strategy  for  generating  hypotheses  and  applied  research  in  intelligence  analysis  is 
needed. 

Heuer  (1999,  p.  95-107)  has  also  written  the  most  extensively  on  a  technique  he 
developed,  Analysis  of  Competing  Hypothesis  (ACH),  to  counteract  analysts’  tendencies  to 
intuitively  select  what  they  consider  to  be  the  most  likely  answer  and  look  at  whether  the 
available  information  supports  the  selected  answer.  The  technique  entails  identifying  possible 
hypotheses  by  brainstorming,  listing  evidence  for  and  against  each,  analyzing  the  evidence  and 
then  refining  hypotheses,  trying  to  disprove  hypotheses,  analyzing  the  sensitivity  of  critical 
evidence,  reporting  conclusions  with  the  relative  likelihood  of  all  hypotheses,  and  identifying 
milestones  that  indicate  events  are  taking  an  unexpected  course. 

The  use  of  brainstorming  in  ACH  is  a  key  feature,  because  in  a  top-down  or  hypotheses- 
driven  process,  the  quality  of  the  hypotheses  is  dependent  on  the  existing  knowledge  and 
experience  of  the  analysts,  since  hypotheses  generation  occurs  before  additional  infonnation 
acquisition  augments  the  existing  knowledge  of  the  problem.  Increasing  the  number  of 
hypotheses  generated  is  consistent  with  Kent’s  view  (1949,  p.  174)  that  “What  is  desired  in  the 
way  of  hypotheses,  whenever  they  may  occur,  is  quantity  and  quality.”  However,  in  contrast  to 
Heuer,  Kent  emphasizes  extensive  information  acquisition  based  on  understanding  the  problem 
( purpose j  before  the  deliberate  generation  of  hypotheses.  He  states,  “. .  .two  things  an 
intelligence  organization  must  have  in  order  to  generate  more  and  better  hypotheses:  (1) 
professional  staff  of  the  highest  competence  and  devotion  to  the  task,  and  (2)  access  to  all 
relevant  data.” 
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ACH  is  widely  cited  in  the  intelligence  literature  as  a  positive  method  for  improving 
analysis.  Heuer  (1999,  p.  109)  describes  the  advantages  of  ACH,  including  that  many  potential 
answers  (hypotheses)  to  the  intelligence  problem  are  considered  and  subjected  to  a  consistent 
approach  for  rejection  or  validation.  He  implies  using  ACH  will  mitigate  confirmation  bias  in 
intelligence  analysts  since  many  hypotheses  are  considered,  but  this  has  not  been  substantiated 
by  research  (Cheikes  et  al.  2004,  p.  1  &  p.  16). 

In  contrast,  Klein  et  al.  (2007,  p.  146)  reference  a  study  by  Rudolph  (2003)  on  medical 
practitioners  showing  extensive  hypothesis  generation  was  “...almost  as  dysfunctional  as 
fixation”  on  a  given  hypothesis.  The  data-frame  theory  (Klein  et  al.  2007)  referenced  earlier 
“...regards  early  commitment  to  a  hypothesis  as  inevitable  and  advantageous,”  permitting 
“...efficient  information  gathering  and  more  specific  expectancies  that  can  be  violated  by 
anomalies,  permitting  adjustment  and  refraining.”  Klein  (‘Improving  Intelligence  Analysis’ 
2007,  p.  46)  also  characterized  keeping  an  open  mind  and  avoiding  premature  consideration  of  a 
hypothesis  as  “. .  .one  of  the  classic  mistaken  beliefs  about  analysis.”  The  central  message  is  not 
to  keep  an  open  mind  or  avoid  having  hypotheses,  since  this  is  not  possible  due  to  the  nature  of 
mental  models,  but  to  keep  testing  one’s  hypotheses  and  judgments. 

Intelligence  Research 

The  analysis  processes  described  (bottom-up,  top-down,  and  sensemaking)  all  require  the 
acquisition  of  facts,  data,  and  infonnation  (the  name  used  to  describe  the  input  to  intelligence 
analysis  continues  to  evolve).  Generally  this  acquisition  phase  is  called  intelligence  research, 
although  some  have  used  the  tenn  to  describe  the  entire  analysis  process  (Clauser  &  Weir  1976). 

Although  Kent  (1949)  does  not  provide  a  description  of  research  in  his  classic  work  on 
intelligence,  he  does  in  Writing  History  (Kent  1967).  The  essence  of  the  Kent’s  research  method 
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is  to  find  all  that  one  can  about  the  topic  being  researched  (Kent  1967,  p.  25)  and  to  develop 
comprehensive  knowledge  of  all  the  applicable  sources  (Kent  1967,  p.  25-34).  Drell  (1957)  later 
provides  a  more  detailed  description  of  intelligence  research  consistent  with  Kent’s. 

The  search  of  infonnation  in  the  bottom-up  approach  is  problem-based,  guided  by  one ’s 
understanding  of  the  problem  under  study  (Kent  1949,  p.  157;  Valtorta  et  al.  2005,  p.  1).  In  the 
top-down  approach,  the  tentative  hypotheses  are  used  as  the  conceptual  framework  for  selecting 
information  (Heuer  2005,  p.  91;  United  States,  pp.  106-107).  Schum  (1987,  p.  16)  is  more 
explicit  about  the  role  of  hypotheses:  “Having  expectations,  in  the  fonn  of  initial  or  preliminary 
hypotheses,  is  commonly  recognized  as  a  necessary  prerequisite  for  efficient  and  productive  data 
search.”  Schum  is  likely  correct  that  searching  for  infonnation  relating  to  a  set  of  hypotheses 
requires  less  time  and  effort  than  searching  for  all  infonnation  relevant  to  the  entire  scope  of  the 
problem  under  analysis.  However,  thorough  problem  exploration  and  purpose  refinement 
followed  by  infonnation  acquisition  based  on  the  problem  will  likely  result  in  more  complete 
acquisition  of  all  the  relevant  evidence.  This  has  significant  implications  for  the  conduct  of  the 
intelligence  process;  research  is  needed  to  fully  understand  these  implications. 

Since  Drell’s  (1957)  treatment  of  intelligence  research,  the  topic  has  received  very  little 
attention.  This  lack  of  attention,  coupled  with  similarly  large  gaps  in  the  intelligence  literature 
on  the  topics  of  information  relevance  and  retrieval,  constitutes  a  major  omission  in  the 
literature,  given  the  role  and  importance  of  information  in  the  analysis  process.  Research  is 
needed  to  address  these  gaps. 

Some  have  ascribed  the  lack  of  treatment  of  intelligence  research  to  the  shift  in  emphasis 
from  long-tenn  analysis  to  current  intelligence  (AFCEA  2005,  p.  6)  Kent  (1949,  p.  196)  warned 
of  the  diversion  of  effort  from  long-tenn  analysis  to  more  current  tasks  and  the  compelling 
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tendency  to  put  analysts  on  this  kind  of  work.  He  does  not  argue  that  it  is  unimportant,  but  that  it 
absorbs  too  much  talent.  He  emphasizes,  “Intelligence  should  have  long  stretches  of 
uninterrupted  time  to  carry  out  long-range  project  which  can  be  done  in  no  other  circumstances.” 
(Kent  1949,  pp.  196-197) 

When  Hilsman  (1981)  surveyed  policymakers  (c.  1956)  about  their  needs  for  intelligence, 
he  found  a  strong  preference  for  current  intelligence  versus  long-term  study.  Hilsman  was  most 
surprised  that  much  of  the  pressure  for  current  intelligence  arose  from  intelligence  personnel. 
Subsequently  many  authors,  with  increasing  frequency,  have  lamented  about  the  emphasis  on 
current  intelligence  and  the  ramifications  of  this  shift  on  long-term  analysis.  The  concern  is 
captured  in  the  following  (Risen  2006,  p.  7): 

“’If  I  had  to  point  to  one  specific  problem  that  explains  why  we  are  doing  such  a  bad  job 
on  intelligence,  it  is  this  almost  single-minded  focus  on  current  reporting,’  observes  Carl 
Ford,  a  former  CIA  analyst  and  former  chief  of  the  Bureau  of  Intelligence  and  Research 
at  the  State  Department.  In  the  1970s,  Ford  adds,  70  percent  to  80  percent  of  CIA 
analysts  spent  their  time  doing  basic  research  on  key  topics;  today,  about  90  percent  of 
analysts  do  nothing  but  current  reporting.  ‘Analysts  today  are  looking  at  intelligence 
coming  in  and  then  writing  what  they  think  about  it,  but  they  have  no  depth  of  knowledge 
to  determine  whether  the  current  intelligence  is  correct.  There  are  very  few  people  left  in 
the  intelligence  community  who  even  remember  how  to  do  basic  research.’" 

Marshalling  of  Evidence 

After  infonnation  is  acquired,  cognitive  processes  are  used  to  extract  evidence  from  the 
information,  to  interrelate  evidence  and  evaluate  various  hypotheses,  and  ultimately  select  a 
hypothesis  or  hypotheses  (depending  on  the  nature  of  the  problem  under  study)  with  associated 


28 


likelihoods.  Despite  the  crucial  nature  of  these  actions,  the  intelligence  literature  is  silent  on  how 
to  do  this  part  of  the  process,  with  one  notable  exception,  the  work  of  David  A.  Schum  and  his 
colleagues.  They  treat  the  complete  spectrum  of  constructing  inference  chains  and  their 
ingredients,  judging  credibility,  marshalling  of  evidence  and  drawing  conclusions  (Schum  1987; 
Schum  1999;  Schum  2001;  Anderson  et  al.  2005).  Schum  addresses  the  many  subtleties  of 
drawing  conclusions  from  massive  amounts  of  evidence,  including  the  role  of  hypotheses, 
categorizing  evidence,  hierarchical  inference,  source  credibility,  and  assessing  inferential  weight 
using  Bayes’  Rule  and  other  less  commonly  referenced  approaches  to  probability  (e.g.,  Shafer’s 
belief  functions  and  Baconian  probability).  Given  that  detennining  the  meaning  of  infonnation 
is  the  core  of  analysis,  Schum’s  work  (1987)  is  a  classic,  foundational  reference  work,  with  day- 
to-day  relevance. 

From  the  wide  range  of  topics  that  Schum  deals  with,  three  are  especially  relevant  for 
providing  insight  to  the  intelligence  literature:  hierarchical  inference,  ancillary  evidence,  and 
assumptions. 

The  concept  of  hierarchical  inference  is  based  on  the  idea  that  inferences  from  evidence 
are  a  necessary  input  to  draw  inferences  from  another  piece  of  evidence.  As  Schum  (1987,  p.  52) 
points  out,  these  inferences  can  be  established  in  either  a  bottom-up  or  a  top-down  approach. 
Drawing  inferences  from  each  item  of  evidence  is  inherent  in  both  bottom-up  and  top-down 
approaches,  and  these  inferences  are  then  used  as  an  input  to  drawing  inferences  from  other 
items  of  evidence.  A  comparable  approach,  albeit  expressed  differently,  has  been  suggested  by 
Connell  (Connell  2007,  p.  13).  He  proposes  that  “Instead  of  trying  to  produce  several  alternative 
explanations  accounting  for  multiple  data  items  simultaneously...,  the  analyst  could  focus  on  a 
single  datum  at  a  time  and  brainstonn  its  implications  for  just  one  aspect  of  an  explanation. . .” 
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This  approach  seems  to  be  conceptually  different  than  the  treatment  of  evidence  described  in  the 
ACH  method. 

Schum  (1987,  p.  93)  defines  ancillary  evidence  as  “evidence  about  evidence”  that  has  an 
indirect  relevancy  to  a  major  conclusion.  Acquiring  ancillary  evidence  requires  different 
strategies.  Infonnation  that  will  serve  as  ancillary  evidence  is  not  likely  to  result  from  either  a 
problem-based  information  acquisition  strategy  or  one  based  on  a  set  of  hypotheses.  Specific 
consideration  must  be  given  to  how  to  derive  that  ancillary  evidence  and  then  acquire  it.  For 
example,  one  must  first  identify  potential  direct  evidence.  Based  on  that  potential  direct 
evidence,  one  must  then  derive  what  elements  of  infonnation  potentially  could  serve  as  ancillary 
evidence  to  provide  insight  on  matters  where  perhaps  the  direct  evidence  is  not  available. 

Specific  approaches  for  the  process  of  then  acquiring  such  ancillary  evidence  are  not  discussed  in 
the  intelligence  literature.  Research  and  resulting  guidance  for  analysts  on  how  to  derive  and 
acquire  ancillary  evidence  is  needed. 

While  there  is  much  discussion  in  the  intelligence  literature  about  the  importance  of 
identifying  assumptions,  no  other  works  address  both  the  scope  of  what  should  be  considered  as 
assumptions  and  offer  the  depth  of  treatment  that  Schum  does.  He  describes  assumptions  as  the 
essential  “glue”  holding  arguments  and  the  supporting  inference  networks  together  (Schum  1987, 
p.  35)  Schum  (1987,  p.  17)  also  points  out  that  differences  in  judgments  regarding  the  meaning 
of  infonnation  are  usually  due  to  the  existence  of  different  assumptions.  Anderson  et  al.  (2005, 
p.  266-276)  provide  a  comprehensive  explanation  of  the  wide  range  of  beliefs  that  should  be 
considered  as  assumptions,  ranging  from  our  personal  “stock  of  knowledge”  or  “common  sense” 
through  scientific  knowledge  that  varies  in  terms  of  reliability.  Muller  (2008,  p.  2)  provides 
illustration  of  this  in  his  discussion  of  how  “...a  person’s  underlying  set  of  beliefs,  assumptions, 
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experiences-even  wisdom-about  how  the  world  works”  affect  judgments  made  by  that  person. 
Muller  uses  the  Red-Blue  political  dichotomy  based  on  graphics  used  in  the  2000  presidential 
election  to  portray  how  worldviews  affect  intelligence  analysis.  A  similar  view  has  also  been 
expressed  regarding  why  people  disagree  so  stridently  about  foreign  policy  (Nau  2007). 

Statements  are  widespread  regarding  the  necessity  of  explicitly  identifying  the 
assumptions  used  in  the  analysis.  These  discussions  do  not  focus  on  the  wide  range  of 
assumptions  that  are  fundamental  to  how  analysts  detennine  meaning.  Consequently,  a  reason 
for  the  apparent  failure  of  the  analysts  to  comply  with  these  directions  can  be  inferred  from  the 
discussion  of  Red-Blue  analysts  (Muller  2008),  as  well  as  the  wide  range  of  assumptions 
identified  by  Anderson  et  al.  (2005).  That  is,  analysts  are  unlikely  to  equate  their  beliefs  to 
assumptions  (Muller  2008).  A  comprehensive,  in-depth  understanding  of  all  the  elements 
discussed  in  Schum’s  treatment  of  inference  and  evidence  is  essential  to  promoting  rigor  in 
intelligence  analysis  and  improving  the  detennination  of  meaning. 

Rigor 

Since  September  11,  2001  and  the  numerous  studies  of  the  failure  and  resulting  intelligence 
reform,  the  importance  of  rigorous  analytic  thinking  has  been  emphasized.  While  there  is  much 
in  the  intelligence  literature  about  rigor,  there  are  few  specifics  on  how  to  improve  it  throughout 
the  analytic  process.  Zelik  et  al.  (2007)  have  conducted  research  to  refine  the  understanding  of 
rigor  in  intelligence  analysis.  They  point  out  that  generic  definitions  of  rigor  often  focus  only  on 
rigid  and  meticulous  adherence  to  processes  with  precise  and  exacting  standards.  Their  research 
indicates  that  expert  infonnation  analysis  is  often  not  rigid,  but  flexible  and  adaptive  to  highly 
dynamic  environments  and  judgments  about  rigor  must  reflect  the  appropriateness  of  fit  between 
analytic  processes  and  contextual  requirements.  Consequently,  “. .  .rigor  is  more  meaningfully 
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viewed  as  an  assessment  of  degree  of  sufficiency,  rather  than  degree  of  adherence  to  an 
established  analytic  procedure.”  (Zelik  et  al.  2007,  p.  1)  Based  on  their  research  and  findings 
from  other  studies  of  professional  intelligence  analysts,  they  have  identified  eight  critical 
attributes  of  analysis  processes  that  contribute  to  assessments  of  rigor  (Zelik  et  al.  2007,  p4). 

With  the  exception  of  addressing  the  problem  definition  and  exploration  steps,  these  attributes 
encompass  the  analysis  process  steps  identified  by  Kent,  Platt,  and  Heuer.  Zelik  et  al.  have  also 
identified  attributes  focused  on  incorporating  perspectives  of  other  domain  experts,  which  are  not 
explicit  steps  in  the  Kent,  Platt,  and  Heuer  processes. 

Structured  analytic  techniques  and  automated  aids  and  tools  have  much  to  offer  relative 
to  the  attributes  of  rigor  identified  by  Zelik  et  al.  (2007).  Potential  contributions  include 
systematic  approaches  for  verifying  information,  identifying  the  source’s  perspective  when 
evaluating  data,  considering  and  understanding  assumptions  and  limitations  of  analysis, 
extracting  and  integrating  diverse  interpretations  of  information,  examining  the  chain  of 
reasoning,  explicitly  identifying  stronger  and  weaker  inferences,  and  exploring  and  weighing 
alternatives.  At  the  root  of  such  techniques  is  the  use  of  probabilities  and  belief  functions  to 
quantify  uncertainty;  explore  alternatives;  evaluate  the  completeness  of  evidence;  and  assess 
relative  likelihoods,  risks,  and  ambiguities  associated  with  specific  states.  These  techniques  can 
also  provide  the  advantage  of  an  observable  trail  of  judgments  that  can  be  used  as  the  basis  for 
assessing  rigor. 

Automated  tools  have  been  and  continue  to  be  developed  with  the  objective  of  improving 
analysis  and  supporting  the  analytic  process,  including  increasing  rigor,  dealing  with  uncertainty, 
and  alleviating  aspects  of  analysts’  workload  (e.g.,  data  overload).  A  considerable  amount  of 
literature  on  these  tool  developments  exists  and,  generally,  most  tool  developments  are  in  two 
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broad  categories:  (1)  data  acquisition  and  retrieval,  and  (2)  hypothesis  generation  and 
evaluation.  To  date,  very  few  developments  have  attempted  to  address  the  continuous  evolution 
of  purpose,  context,  and  the  analyst’s  knowledge— all  key  factors  dynamically  shaping  the  mental 
model— into  processing  information  and  the  conduct  of  the  intelligence  process  in  its  entirety. 

Recently  new  approaches  have  emerged,  based  on  sensemaking  research,  to  develop 
automated  work  environments  that  consider  and  support  the  analysis  process  in  a  holistic  way — 
as  an  end-to-end  coherent  whole  instead  of  treating  only  specific  elements  of  the  process  without 
the  benefit  of  the  context  of  the  other  interdependent  elements  (Hollywood  et  al.  2004;  Eggleston 
et  al.  2005;  Pioch  &  Everett  2006)  The  holistic  approach  of  these  developments  is  in  contrast  to 
the  myriad  of  prior  and  ongoing  developments  that  focus  on  supporting  only  a  specific  element 
of  the  analysis  process  (e.g.,  data  mining,  visualization,  Bayesian  belief  networks,  social 
networks,  ACH,  inference  networks). 

As  Zlotnick  stated  (1972,  p.  42),  “The  very  best  that  intelligence  can  do  is  make  the  most 
of  the  evidence  without  making  more  of  the  evidence  than  it  deserves.”  Automated  work  support 
aids,  tools,  and  structured  analytic  techniques  provide  a  variety  of  means  to  increase  overall  rigor 
in  the  identification  and  treatment  of  evidence,  thereby  improving  the  determination  of  meaning 
throughout  the  entire  analysis  process.  If  chosen  appropriately  considering  the  holistic  nature  of 
the  intelligence  issue  under  study,  they  can  contribute  rigor  even  though  pure,  sterile  objectivity 
is  unobtainable  due  to  the  nature  of  determining  meaning.  While  rigor  is  often  mentioned  in  the 
literature  and  specific  tools  and  techniques  received  a  great  deal  of  attention,  the  literature  and 
the  practice  of  analysis  would  be  enriched  by  additional  research  on  understanding  the  various 
attributes  of  rigor  in  intelligence  analysis.  In  addition,  research  is  needed  on  implementing 
methods  for  assessing  those  attributes  in  actual  intelligence  work  environments,  as  well  as  on  the 
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development  and  use  of  holistic  automated  work  environments  that  incorporate  the  strengths  of 
the  various  processes  and  approaches  in  the  intelligence  literature  to  attain  such  rigor. 

Observations 

Because  the  world  has  changed  a  great  deal  since  the  classic  writings  of  intelligence  analysis, 
some  may  be  tempted  to  dismiss  them  out  of  hand  as  irrelevant  or  outdated.  Yet  all  well 
established  professions  (e.g.,  law)  have  an  established,  foundational  literature  sometimes 
centuries  old  which  endures.  Those  professions  well  advanced  in  terms  of  their  development 
also  have  literatures  which  have  expanded  the  foundation  well  beyond  the  initial  principles  to 
deal  with  the  full  complexities  of  the  core  aspects  of  the  profession. 

While  much  as  been  written  about  intelligence  since  the  1940s  and  1950s,  and  there  has 
been  much  written  on  various  activities  that  intelligence  analysts  must  do,  the  literature  has  not 
advanced  on  the  core  aspect  of  determining  meaning  from  infonnation  to  address  the  full  range 
of  complexities  in  intelligence  analysis.  After  the  initial  classic  works,  the  discourse  on  this  has 
largely  disappeared.  That  is  not  to  say  that  there  have  not  been  new  ideas  put  forth  relative  to 
how  to  better  deal  with  certain  individual  components  of  the  work  (e.g.,  special  techniques, 
technologies  to  manipulate  more  information,  etc.).  Yet,  a  comprehensive  approach  to  move 
forward  and  tackle  the  complexities  of  the  work  has  not  fully  emerged. 

In  some  respects  intelligence  analysis  has  been  addressed  in  a  way  consistent  with  the 
standard  definition  of  analysis,  breaking  up  of  the  whole  into  parts,  rather  than  a  focus  on  the 
synthesis  of  the  profession  “as  to  form  a  whole.”  In  a  few  cases,  works  such  as  those  on 
sensemaking  have  “re-introduced”  the  notion  of  addressing  the  core  of  the  phenomena  of 
intelligence  analysis,  its  very  essence,  to  advance  the  practice  of  intelligence  analysis  and  its 
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ability  to  deal  with  the  aspects  of  complexity,  uncertainty,  and  dynamic  environments  that 
traditional  scientific  methods  have  not  been  able  to  fully  address. 

While  no  single  approach,  over  time,  fully  dominates  the  other,  embedded  in  the 
literature  are  the  strengths  of  the  various  approaches.  Key  elements  are  a  holistic  framing  and 
exploration  of  the  problem  (as  indicated  early  on  by  Kent  and  Platt),  systematic  data  collection 
relative  to  the  problem  and  purpose,  rigorous  treatment  of  the  data  and  evidence,  generating 
hypotheses  based  on  knowledge,  and  repeated  testing  and  retesting  of  one’s  hypotheses. 

Since  the  beginning  of  modern  intelligence  analysis,  there  have  been  concerns  about  the 
pressures  of  time,  the  experience  level  of  analysts,  and  relevance  to  customers.  There  ultimately 
are  no  cognitive  shortcuts  to  using  the  best  of  the  classic  approaches.  The  price  of  doing  so  will 
always  be  paid  later,  if  not  now.  Tools  and  resources  can  provide  essential  support  to  facilitate 
thinking  and  reduce  cognitive  burden,  as  can  constructive,  enabling  management  and 
organizational  practices.  Imperative,  though,  is  the  need  for  a  concerted  focus  on  the  core  aspect 
of  analysis:  how  do  analysts  determine  meaning  and  what  can  aid  them  in  doing  so.  While 
times  and  context  change  and  the  peripheral  issues  are  adjusted,  as  with  more  established 
professions  (medicine,  law,  etc),  the  phenomena  of  analysis  maintains  a  stability  at  the  core.  The 
challenge  for  intelligence  studies  on  analysis  is  to  focus  on  the  core  aspect  of  analysis, 
determining  meaning,  in  a  holistic  way  to  address  gaps  in  understanding  and  in  the  literature, 
providing  the  foundation  to  implement  in  practice. 
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